The role of the mass vaccination programme in combating the COVID-19 pandemic: An LSTM-based analysis of COVID-19 confirmed cases

The COVID-19 virus has impacted all facets of our lives. As a global response to this threat, vaccination programmes have been initiated and administered in numerous nations. The question remains, however, as to whether mass vaccination programmes result in a decrease in the number of confirmed COVID-19 cases. In this study, we aim to predict the future number of COVID-19 confirmed cases for the top ten countries with the highest number of vaccinations in the world. A well-known Deep Learning method for time series analysis, namely, the Long Short-Term Memory (LSTM) networks, is applied as the prediction method. Using three evaluation metrics, i.e., Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE), we found that the model built by using LSTM networks could give a good prediction of the future number and trend of COVID-19 confirmed cases in the considered countries. Two different scenarios are employed, namely: ‘All Time’, which includes all historical data; and ‘Before Vaccination’, which excludes data collected after the mass vaccination programme began. The average MAPE scores for the ‘All Time’ and ‘Before Vaccination’ scenarios are 5.977% and 10.388%, respectively. Overall, the results show that the mass vaccination programme has a positive impact on decreasing and controlling the spread of the COVID-19 disease in those countries, as evidenced by decreasing future trends after the programme was implemented.


Introduction
The coronavirus disease  has undoubtedly become one of the most memorable events of the year 2020. A major public health issue, COVID-19 is believed to have been first detected in late 2019 in China [1][2][3][4]. Due to its massive and progressive spread, in the early 2020 the disease was declared a global pandemic by the World Health Organization [5,6]. The latest updates on COVID-19, dated February 14, 2022, show that there are more than 410 million people around the world who have been infected by this disease, with almost 5.9 million of them having died [7].
To combat the pandemic, a mass vaccination programme has been started globally since 2020. The United Kingdom (UK) has become the first country in the world to administer its citizens a fully tested COVID-19 vaccine on December 8, 2020 [8]. Soon enough, many other countries followed the UK's lead to approve and administer COVID-19 vaccines to their people. As reported by BBC [9] from its 'Our World in Data' source [10], the top ten countries with the highest total number of vaccinations carried out to date (February 2022)  The World Health Organization (WHO) together with Gavi and the Coalition for Epidemic Preparedness Innovations (CEPI), has also promoted the COVAX initiative. As one of the three pillars of the Access to COVID-19 Tools Acceleratordiagnostics, treatments, and vaccines -COVAX has focused on the third one [11,12]. It aims to accelerate the development and manufacture of COVID-19 vaccines and ensure that they are equitably distributed around the world [12,13]. Therefore, with the mass vaccination programme and the COVAX initiative, we may have higher hopes of ending the COVID-19 pandemic sooner and returning to our normal lives before the pandemic occurred.
Despite the future prospects, one simple question remains: Will the mass COVID-19 vaccination programme result in a decrease in confirmed COVID-19 cases? The successful distribution and administration of COVID-19 vaccines across the Globe should decrease the number of COVID-19 confirmed cases. Therefore, to get an insight into this question, in this study we aim to predict and analyse the COVID-19 confirmed cases before and after the mass vaccination programme rolled out. A Deep Learning method, known as the Long Short-Term Memory (LSTM) networks, will be employed as the main prediction method. LSTM was developed to tackle the limitations found in the Recurrent Neural Network (RNN) method, which suffers from short-term memory, vanishing and exploding gradient [14]. As one of the advanced forecasting methods commonly used in time series analysis and other applications with astounding results [15], it could be classified as a black-box model [16].
Various Deep Learning methods, such as Generative Adversarial Networks (GANs), Extreme Learning Machine (ELM), and LSTM, have been employed in COVID-19 research [17] and in spread forecasting for epidemiology [18]. In particular, the LSTM method has been used to predict the COVID-19 epidemic transmission and trends. Chimmula and Zhang [19], for example, developed a forecasting model for the COVID-19 outbreak in Canada by using LSTM networks. With a relatively small amount of data, they predicted that the ending point of COVID-19 outbreak in Canada would be around June 2020 [19]. Wang et al. [20] also used the LSTM method, which was further improved by using a rolling update mechanism and Diffusion Index to predict the COVID-19 epidemic trends in Russia, Peru, and Iran. They predicted that the epidemic in Peru would peak around early December 2020, while the number of positive cases per day in Iran would fall below 1000 by mid-November 2020, in contrast to Russia, which was predicted to have an increment of more than 2000 cases per day by early December 2020 [20]. Another implementation of the LSTM model has been introduced by Pathan et al. [21]. They used LSTM to predict the future mutation rate of SARS-CoV-2, the novel coronavirus that caused the COVID-19 pandemic. They concluded that if more patient data had been made available in an updated time period, the proposed model could be used to predict the mutation rates of this virus on a daily basis [21]. Table 1 summarises several studies that used Machine and Deep Learning methods to predict COVID-19. Although LSTM has been widely used to predict COVID-19 future trends, to the best of our knowledge, this is the first study to apply LSTM networks in the prediction and analysis of COVID-19 confirmed cases before and after the mass vaccination programme was implemented, with a particular focus on the top ten countries with the highest total number of vaccination doses delivered. LSTM was chosen because it has been widely accepted and successfully applied in a variety of cases, particularly in the time series domain. The successful application of the proposed LSTM networks in predicting COVID-19 confirmed cases before and after the mass vaccination programme was implemented could aid decision-makers in devising better pandemic management strategies.
Artificial intelligence (AI) is a new paradigm for healthcare systems, and it is important to note that intelligent machine learning algorithms can be used to analyse COVID-19 data and provide information for decision-making processes. This implies that tools powered by AI can aid in predicting the number of confirmed COVID-19 cases. A fundamental requirement is the availability of sufficient data to train the respective models. Earlier in the pandemic, the majority of AI-powered tools utilised by previous studies to forecast and predict the pandemic were limited to proof-of-concept models. However, as more and more data are generated every day, this presents the opportunity to reevaluate the robustness of existing algorithms.
A large number of algorithms are created frequently. And while we recognise the importance of developing new and perhaps better algorithms, it is also important that we maintain a balance by utilising what we already have that has been proven effective. In this regard, then, our work contributes to the existing body of knowledge. The fact that we use a well-known method with a track record of proven robustness (i.e., LSTM) to predict the number of confirmed COVID-19 cases and the future trend is an advantage that helps to counteract the phenomenon known as COVID-19, which is still poorly understood. Overall, we were able to demonstrate that the mass vaccination programme contributes to reducing and controlling the spread of the COVID-19 disease in those countries, as indicated by the decreasing future trends after the mass vaccination programme was administered. This can help relevant decision-makers make better practical decisions and take appropriate actions or measures to contain or limit the coronavirus's spread.

Materials and methods
In this section, we first describe the data source being used in this study. Next, we explain the basic concept of LSTM networks that are used as the main prediction method in this study, followed by a brief explanation of several evaluation metrics used.

Data source
The main data source of COVID-19 confirmed cases used in this study was collected from a GitHub data repository, which is operated and maintained by the Johns Hopkins University (JHU) Center for Systems Science and Engineering [29]. This repository is updated and curated by a team of scientists at JHU since the early time of COVID-19 outbreak, and the data visualisation is depicted in an online real-time interactive dashboard [30]. It can be accessed publicly and has been widely used in many publications [16,19,20].
We used the global time series data of COVID-19 confirmed cases, which was named as 'time_series_covid19_confirmed_global.csv' and taken on February 14, 2022 (last recorded data on February 12, 2022). The document consists of more than 280 regions' data, but we will focus on the ten countries or regions with the highest total vaccinations volume up to date [9,10], namely China, India, the United States, Brazil, Indonesia, Japan, Pakistan, Vietnam, Mexico, and Germany. Table 2 presents the summary statistics of the dataset used in this study.

LSTM networks
Long Short-Term Memory (LSTM) is an advanced soft computing method, which was derived from the Recurrent Neural Network (RNN). RNN itself actually is one of the many types of Artificial Neural Networks (ANN) methods, which was proposed to overcome the ANN's disadvantage in handling the time correlation in data sequence. It adds canonical connections to neurons in the networks, so that the sequence-to-sequence mapping between input and output data can be built by RNN [31]. Unfortunately, classical RNN still struggles with the long-range dependencies, suffering from exploding gradient or, in contrast, from vanishing gradient, which limits its ability to learn the long-term temporal correlations [32]. Therefore, LSTM was introduced by Hochreiter and Schmidhuber (1997) to overcome this limitation by using memory cells [33]. These cells are self-connected and store the networks' temporal state by using a three-gate mechanism, composed of the input gate, the output gate, and the forget gate [33]. Fig. 1 depicts an LSTM cell, which contains all those three gates and the cell state [34,35]. LSTM gates are simply used as a way to control how much information can be passed. Commonly, they are composed of a sigmoid neural network layer and a pointwise multiplication operation. Forget gate is used to forget the information in the cell state selectively, while the input gate is used to determine what new information will be stored in the current cell state. Lastly, the output gate is used to find what value we want to output [31].
The first part of the LSTM cell is the forget gate. It is used to control the magnitude to forget the hidden state of the previous cell and it can be expressed as shown in Eq 1: where f t denotes the forget gate value at the current cell, which ranges from 0 (completely forget) to 1 (completely keep), and W f , U f are the weights of the networks; b f is the bias variable value, h t− 1 is the prior hidden state value, and x t is the new input value at the current cell. Next, to update the cell state, we use the input gate. There are two actions that will be taken in this step. First, for the input gate, we pass the prior hidden state value (h t− 1 ) and the current input value (x t ) into a sigmoid function as shown in Eq [2]. The resulting value of the input gate (i t ) decides the magnitude of the new information that will be kept in the current cell, where 0 means 'completely ignore' and 1 means 'completely keep'. Second, we also pass the prior hidden state value (h t− 1 ) and the current input value (x t ) into the tanh function to help regulate the network as shown in Eq [3]. Similarly, when it comes to the forget gate, there are some weights of the networks and bias values involved in this step, as denoted by At this point, we have enough information to calculate the (current) cell state (C t ). The previous cell state (C t− 1 ) will be pointwise multiplied with the forget vector (f t ). Then, we do a pointwise addition with the output from the input gate (i t ), which has been pointwise multiplied with the cell candidate value (C t ), as shown in Eq [4].: In the last step, we use the output gate to decide what the next hidden state should be (i.e., the current hidden state value, h t ). First, we pass the prior hidden state value (h t− 1 ) and the current input value (x t ) into the sigmoid function as shown in Eq [5]. Here, W o , U o , and b o are the corresponding networks weights and bias values for the output gate. Then, we pass the newly found cell state (C t ) to the tanh function and pointwise multiply the output with the sigmoid output from the output gate (o t ) as shown in Eq [6]. The output from this last process is the current hidden state value (h t ), which will be passed together with the current cell state (C t ) to the next time step. Fig. 1. An LSTM cell and its gates [35].

Evaluation metrics
Three different prediction error criteria will be used as the evaluation metrics in this study. These are the Mean Absolute Error (MAE), the Root Mean Squared Error (RMSE), and the Mean Absolute Percentage Error (MAPE). The first two show the degree of error in a unit value, while the last one shows the degree of error in a percentage value. As described by Shahid et al. [33] and Hansun et al. [36,37], all those three criteria can be expressed as shown in Eq. [7], Eq. [8], and Eq. [9], respectively: where n is the total number of data, Y t is the actual value, and F t is the predicted value. Moreover, we will also use the popular Mean Squared Error (MSE) criterion in the calculation of the loss function during the LSTM networks training. The formula for MSE is shown in Eq10 [38].

Results and discussion
In this section, first we explain the data splitting and the pre-processing of the ten countries considered in this study before we move to the implementation and prediction results of COVID-19 confirmed cases by using LSTM networks. The analysis and discussion of the effect of the mass vaccination programme in these countries will be provided in the last part of this section.

Data splitting, pre-processing, and model building
There are ten countries considered in this study and these are those that have the highest total number of vaccinations [9,10]: China, India, the United States, Brazil, Indonesia, Japan, Pakistan, Vietnam, Mexico, and Germany. First, we will plot the number of COVID-19 confirmed cases in these countries from the first available data on January 22, 2020 to the last recorded data on February 12, 2022 as shown in Fig. 2. We will consider those data as the 'All Time' data that show the number of COVID-19 confirmed cases for each country even after the mass vaccination programme started. Next, for simplicity, we assume that all countries considered have the same effective date for the mass vaccination programme having taken effect in controlling the disease, i.e., on February 1, 2021. Therefore, the available data for each country since January 22, 2020 until February 1, 2021 will be considered as the 'Before Vaccination' data that show the number of COVID-19 confirmed cases before the mass vaccination programme had taken effect in those countries.
Next, for the data splitting process, we use the 80:20 ratio to split the data of each considered country into training and test set. Moreover, we also use 14 timestamps (span), meaning that the 14 last days will be used to consider the future number of COVID-19 confirmed cases for each country. Table 3 shows the splitting numbers of each country's data considered in this study.
After the data splitting process, we conduct the data normalisation process (feature scaling) by using the MinMaxScaler transformation method. Then, to incorporate the defined timestamps, we create a new function called 'create_dataset'. Lastly, we convert the data shape into a 3D array shape, which is the accepted data shape by the LSTM model in the Keras library.
For the model building process, we use a well-known Deep Learning package for Python, namely Keras. It runs on top of the TensorFlow Machine Learning platform. Several modules from Keras are used to build the LSTM networks in this study, i.e., 'Sequential' to initialise the neural networks, 'Dense' to add a densely connected neural network layer, 'LSTM' to add the Long Short-Term Memory layer, and 'Dropout' to add a dropout layer to prevent overfitting. In summary, we build five-layer neural networks comprised of two LSTM layers, two Dropout layers, and one Dense (output) layer. Interested readers may find the source code and data used in this study in the GitHub repository at https://github.com/senghansun/COVID-19-with-LSTM.

Prediction results and analysis
In this section, we describe the prediction results of COVID-19 confirmed cases for all the ten countries of interest by using a Deep Learning method, namely, the LSTM networks. We divide the prediction results into two categories, one for the 'All Time' data and another one for the 'Before Vaccination' data. 'All Time' results show the prediction results since the first available data to the last recorded data of each country after the mass vaccination programme started. Meanwhile, 'Before Vaccination' results show the prediction results since the first available data for each country to the cut date when we assumed the mass vaccination programme should have taken effect, i.e., on February 1, 2021. Figs. 3 and 4 show the prediction results for Indonesia of both categories, respectively. Meanwhile, the complete prediction results for all countries are provided in the supplementary file (S1).
As previously stated, we also evaluate the prediction results of COVID-19 confirmed cases for each country by using three different evaluation metrics, namely, MAE, RMSE, and MAPE. Table 4 shows the corresponding evaluation metrics results on the test set for each country.
As it can be inferred from Table 4 and from the prediction results plots in Figs. 3 and 4, LSTM networks could predict the number of COVID-19 confirmed cases quite well. LSTM networks work particularly well especially in the cases of Pakistan, Mexico, and Japan for 'All Time' scenario, and India, Vietnam, and Brazil for 'Before Vaccination' scenario, for which the lowest prediction error results based on the evaluation metrics are achieved. In the regression task, lower MAE, RMSE, and MAPE values indicate higher prediction accuracy [39,40]. Furthermore, the average MAPE scores for 'All Time' versus 'Before Vaccination' are 5.977% and 10.388%. Hence, the prediction results using 'All Time' data scenario have better accuracy level, mostly impacted by higher number of time series data available to be used in the model development.
In general, Vanilla RNN may provide lower MAE and RMSE scores than LSTM in both the 'All Time' and 'Before Vaccination' scenarios. It does, however, provide a much higher MAPE score than LSTM networks. Using Vanilla RNN, the average MAPE scores for 'All Time' and 'Before Vaccination' are 7.772% and 19.305%, respectively. Again, the prediction results based on 'All Time' data have better accuracy level than the 'Before Vaccination' scenario. When the average MAPE scores for LSTM and Vanilla RNN are compared, it is clear that LSTM has lower MAPE scores for both the 'All Time' and 'Before Vaccination' scenarios than Vanilla RNN (5.977% vs 7.772% and 10.388% vs 19.305%, respectively). As a result, it is possible to conclude that the proposed LSTM networks outperform the Vanilla RNN in terms of MAPE.
Moreover, we also tried to find the future trend projection of COVID-19 confirmed cases in each country considered by comparing the future prediction result with the last known data record (one period ahead). Table 6 shows the future prediction as well as the trend percentage for each country both for the 'All Time' and 'Before Vaccination' data using the proposed LSTM networks.
Based on the prediction results, most countries in both categories have a downward trend for the number of COVID-19 confirmed cases. The only two exceptions are posed by India with an upward trend of +0.243% and Mexico of +0.079% for the 'All Time' data; while for the 'Before Vaccination' data, the only exception is posed by Germany (+32.697%). However, as we can see from the results, the mass vaccination programme could help in controlling the pandemic, even for those countries that have increased trend projections. The prediction is that Germany, for example, will have quite a big increasing number of confirmed cases in the future if they do not start the mass vaccination programme (#16, 443, 102, +32.697%), while they are projected to have a downward trend when they have started the mass vaccination programme (#10, 336, 484, − 16.584%).
In general, the mass vaccination programme has a positive effect in terms of contributing to decreasing and controlling the spread of the COVID-19 disease in most countries considered. We can easily see that by comparing the future trend results for each country, both for 'All Time' (when the mass vaccination programme has been started and considered to have taken effect) and 'Before Vaccination' data. However, out of the ten countries considered, two of them have shown slightly different results. Both India and Mexico have a greater decreasing trend for 'Before Vaccination' than for 'All Time' data, which means that the mass vaccination programme seems not to be showing any better effect than if the programme had not been administered. This finding might be rooted in several causes, such as the slow government response to handle the pandemic on the early date, the improper handling of the mass vaccination programme by related stakeholders, the increasing in testing capacity on the recent date, and even the lack of community support for the success of the mass vaccination programme. Moreover, it is worth noticing that a new COVID-19 variant, named Omicron, has emerged since November 2021 [41] and affected a great number of people worldwide. It even predicted to be the root of the next wave of COVID-19 outbreak in several countries [42,43]. Without the mass vaccination program, a more severe catastrophe caused by the disease may happen.

Conclusions
The Coronavirus Disease 2019 (COVID-19) has struck us for more than two years since it was declared a global pandemic by the World Health Organization (WHO) in March 2020 [5]. It has affected every aspect of our lives, with more than 410 million people around the world having been infected by this disease and almost 5.9 million of them having died (as of February 14, 2022). As a response to this major public health threat, the mass vaccination programme has been started and administered in many countries around the world since the end of 2020.
In this study, we have aimed to investigate whether the COVID-19 mass vaccination programme really works in terms of contributing to decreasing and controlling the spread of the COVID-19 disease. Therefore, we tried to predict the future number and trend of COVID-19 confirmed cases for the ten countries with the highest number of vaccinations to date, namely, China, India, the United States, Brazil, Indonesia, Japan, Pakistan, Vietnam, Mexico, and Germany. We grouped the recorded data into two categories, i. e., the 'All Time' data and the 'Before Vaccination' data. Then, by using a well-known Deep Learning algorithm, i.e., the Long Short-Term Memory (LSTM) networks, we built a model for each category of data and used them to predict the future number and trend of COVID-19 confirmed cases for each country.
Based on the experimental results, we found that the LSTM networks model could be used to predict the future number and trend of COVID-19 confirmed cases quite well in most countries considered. The average MAPE scores for 'All Time' versus 'Before Vaccination' scenarios are 5.977% and 10.388% respectively. We also found that the mass vaccination programme has a positive effect in terms of contributing to decreasing and controlling the spread of COVID-19 disease in those countries. The only exception is represented by India and Mexico, with both countries having a greater decreasing trend when we predicted the data using the 'Before Vaccination' model. Some factors might cause this finding, such as the slow government response, the improper handling and administration of the mass vaccination programme, the increasing tracing number, the lack of community support for the success of this programme, and even the impact of the new COVID-19 variant named Omicron. Future studies on the barriers to the mass vaccination programme could be taken to answer this question and correlate this finding in more detail.
There are several limitations in our study. Firstly, we applied relatively simple five-layers LSTM networks in predicting the future confirmed cases of COVID-19. We did not put our focus in the optimization and introduction of a new and better prediction model, but rather on the applicability of a well-known Deep Learning method, i.e., the LSTM networks, in predicting confirmed cases 'before' and 'after' the mass vaccination programme rollout. Another limitation is on performance metrics used in this study. We only use three popular error measurement criteria, namely MAE, RMSE, and MAPE, which could not directly measure the trend movement from the prediction results. Directional Statistics (DS) as has been used in several studies [39,40] can be used to assess this trend more accurately.
Given that the future is unpredictable, present predictions must be viewed critically. Nonetheless, a more precise estimate of the number of confirmed COVID-19 cases is essential for optimising available resources and slowing or stopping the pandemic's progression. In addition, our findings can be used to encourage the general public to consider and adhere to the vaccination measures mandated by local and national authorities to halt the pandemic's spread. In this regard, we hope that the present paper can aid a variety of stakeholders in their decision-making processes, thereby facilitating the implementation of appropriate measures to prevent the spread of COVID-19. Overall, this has significant implications for practice, as it would allow policymakers and healthcare providers to plan and determine where to deploy resources.

Declarations
Ethics approval and consent to participate Not applicable.

Consent for publication
Not applicable.

Availability of data and materials
The dataset used and/or analyzed during the study together with the code are available at https://github.com/senghansun/COVID-19-with-LSTM.

Author contribution statement
Seng Hansun: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper. Vincent Charles; Tatiana Gherman: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of interest's statement
The authors declare no conflict of interest.